Attribute Managing Transformers
The handling/managing of attributes is one of the most common operations carried out in FME. Users need to set attributes, edit them, and use them within most workspaces.
The main attribute-management tasks and the transformers that can be used are as follows:
Task | Transformers |
Create Attributes | AttributeCreator, AttributeManager |
Set Attribute Values | AttributeCreator, AttributeManager |
Remove Attributes | AttributeKeeper, AttributeManager, AttributeRemover, BulkAttributeRemover |
Rename Attributes | AttributeManager, AttributeRenamer, BulkAttributeRenamer |
Copy Attributes | AttributeCopier, AttributeCreator, AttributeManager |
Sort Attributes | AttributeManager |
Change Attribute Case | BulkAttributeRenamer |
Add Prefixes/Suffixes | BulkAttributeRenamer |
Note :-Many of these transformers can carry out similar operations, and we can see that the AttributeManager does so many tasks we can use it almost exclusively.
Creating and Setting Attributes
Creating attributes and setting a value are probably the primary attribute functions used within FME. When an attribute is created, its value can be set in a number of ways.
The transformers capable of creating an attribute - and setting its value - are:
- AttributeCreator
- AttributeManager
Note: The AttributeCopier and AttributeRenamer transformers can set an attribute value, but only where the attribute doesn't already exist.
1) The AttributeManager
For most operations we'll concentrate on the AttributeManager, so here is a quick overview of that transformer.
The AttributeManager parameters dialog has a number of fields: Input Attribute, Output Attribute, Attribute Value, and Action. Uniquely among attribute-handling transformers, it is automatically filled in with the details of the attributes connected to it:
The action field can be set by the user but is also set automatically when a change is made to the other fields.
Manually Create an Attribute
By entering a new attribute name into the Output Attribute field, it will be created in the output.
The text <Add new Attribute> highlights where a new attribute can be created. By default, when the Attribute Value field is empty, a new attribute has no value.
Set a Fixed Attribute Value
A fixed (or constant) value for an attribute can be created by simply entering a value into the Attribute Value field:
Here, for example, a new attribute called City is being given a fixed value of Vancouver.
However, also note that the existing attribute NeighborhoodName is also being assigned a fixed value. It has been given the value "Kitsilano." Notice how by entering a value into that field, the Action field has automatically changed from "Do Nothing" to "Set Value."
Constructing Attributes
Besides constant attribute values, FME also allows you to construct values using string manipulation and arithmetic calculations. This procedure is achieved by clicking on the arrow in the Attribute Value field and selecting either Open Text Editor or Open Arithmetic Editor:
This method is advantageous because the attribute no longer needs to be a fixed value: it can be constructed from a mix of existing attributes, parameters, and constants.
Text Editor
The text editor - as you would expect - allows you to construct a text value. It includes all the usual string-handling functionality you would need, such as concatenation, trimming, padding, and case-changing.
The text editor looks like this:
Here the user is constructing an address string by concatenating various existing attributes with some fixed characters (the commas).
The @Value() part is simply an FME function that returns the value of the named attribute.
Notice the menu on the left-hand side. Existing attributes are listed here and were added into the string by double-clicking them. Also, notice the other menu options. Maybe the most important for text are String Functions:
These are the functions that can be used to manipulate the strings being used. For example, here the user is making sure the attributes being used are trimmed when used:
Other functions allow you to calculate results for various date operations; for example, you can determine the interval between two dates in a number of units, you can add or subtract time from a date, or you can change a date from one structure to another.
Arithmetic Editor
The arithmetic editor is much the same as the text editor, except that whatever is entered into the dialog will be evaluated as an arithmetic expression, and a numeric result returned:
Here the user is calculating the monthly number of visitors to a park by dividing the annual number of visitors by 12 (twelve). As with the text editor, existing attributes and arithmetic functions were obtained from the menu on the left-hand side.
FME Feature Functions
One other item in the menu of both text and arithmetic editors is FME Feature Functions:
These are functions that reach into the very heart of FME's core functionality. They are the building blocks that transformers are built upon; basic functionality that can return values to the editor.
For example, the @Area() function returns the area of the current feature (assuming it is a polygon). @CoordSys() returns the coordinate system. They are the functional equivalent of the AreaCalculator and CoordinateSystemExtractor transformers.
@NumCoords() returns the number of vertices in the current feature.
Some functions return strings; others return numeric values. Therefore, the available functions vary depending on whether the text or arithmetic editor is being used. In the screenshot above, the text editor functions are on the left and the arithmetic editor functions on the right. The text editor can use either text or numeric values; the arithmetic editor can only ever accept numeric values.
FME Feature Functions are useful because they allow you to build processing directly into the AttributeManager, instead of using a separate transformer.
Replacing Other Transformers
Integrated text and arithmetic editors provide a great benefit for workspace creation. They allow attribute-creating functions to be carried out directly in a single transformer.
For example, the AttributeManager text editor can be used as a direct replacement for the StringConcatenator and ExpressionEvaluator transformers.
The AttributeManager could also replace the StringPadder and AttributeTrimmer transformers, albeit with a little less user-friendliness.
If FME Feature Functions are used inside the editor, this transformer could also technically replace transformers such as the AreaCalculator, LengthCalculator, DateTimeStamper, VertexCounter and many more.
This feature is usually a good thing. Workspaces will be more compact and well-defined when as many peripheral operations as possible are directly integrated into a single transformer. However, because it's possible for an AttributeManager to be carrying out many, many operations, it is also more important to use Best Practice and ensure it has the proper annotation.
If an AttributeManager is not properly annotated, it isn't possible to determine from looking at the Workbench canvas what action it is carrying out!
Note that incorporating functions into a single transformer is not intended as a way to improve performance. For example, replacing twenty (20) transformers with twenty functions inside one AttributeManager transformer - all carrying out the same task - is unlikely to make the workspace faster overall; plus the AttributeManager will naturally become slower with each new function that is added to it.
Conditional Attribute Values
Conditional Attribute Values are a tool that can be used to replace many existing transformers of the same type.
Transformer-Based Attribute Mapping
Features can be divided within a workspace using transformers in a process called Conditional Filtering, and attributes can be set or created based on these divisions. Here an author divides data into neighborhoods using a postcode attribute:
However, a large number of AttributeManager transformers like this is not a good idea. It causes the workspace to be bloated; hard to navigate and harder to edit. It does not scale particularly well either; since each value needs a separate TestFilter port and AttributeManager combination, it's easy to imagine the difficulties involved with - say - more than 50 values!
One solution is to use a simple AttributeValueMapper transformer. However, that transformer only permits a single, simple condition, such as X=Y. If a more advanced set of conditions is required, then the preferred solution is Conditional Attribute Values.
What are Conditional Attribute Values?
Conditional attribute values are when, instead of creating a set of conditions and values as separate objects in a workspace, the author sets both condition and value inside a single transformer.
The option for conditional attributes is found in the drop-down dialog on most transformer parameters. In the AttributeManager, it appears like so:
In the above screenshot, a workspace author is creating a new attribute called NEIGHBORHOOD. The values for NEIGHBORHOOD are conditional upon other attribute values and - in this example - are set up like this:
Like the AttributeValueMapper, a series of conditions (left) map to different values (right). However, in contrast to the AttributeValueMapper, this dialog allows much more complex conditions than a simple 1:1 mapping. That’s because full test capabilities are built into this dialog.
The conditions are defined by double-clicking in the Test Condition field to open up a Tester-style dialog. Both the condition and the output value can be set within this dialog:
When the conditions are set than the original dialog – in this case, an AttributeManager – looks like this, with the number of conditions defining the number of possible values:
When to use Conditional Attribute Values?
Conditional attribute values are great for when you need to map (or set) an attribute in relation to the value of an existing attribute, and when the conditions are more complex than can be handled in a simple AttributeValueMapper (or AttributeRangeMapper) transformer.
In essence, conditional values are like a combination of TestFilter and AttributeCreators in the range of functionality that they include.
Adjacent Feature Attributes
Normally a feature in FME is self-contained. It might get processed as a group at some point, but other than that it doesn’t have any relationship to other features in the workspace.
However, in some cases, the ability for a feature to access the attributes of other features is quite useful.
For example, take a tabular dataset of coordinates recorded as follows:
X | Y |
0.0 | 3.0 |
3.2 | 0.0 |
-3.2 | 0.0 |
0.0 | 3.4 |
4.2 | 0.0 |
In this case each row is not an absolute coordinate; instead, it is an offset from the previous one. Therefore, to calculate the true coordinates, each feature needs to know the coordinates of the previous feature, so that it can apply the offset.
This sort of scenario is catered for by Adjacent Feature Attributes in FME.
Adjacent Feature Functionality
Adjacent Feature functionality is activated by checking the box labeled Enable Adjacent Feature Attributes in an AttributeCreator or AttributeManager transformer:
This opens up a section of dialog in which the author can specify how many features preceding the current feature, or how many features that succeed it, should be made available. In the above screenshot attributes from the previous feature will become available.
Using Multiple Feature Attributes
The simplest way to make use of the attributes retrieved from prior/subsequent features is through the text or arithmetic editors, where the list of feature attributes has an expandable section for prior and subsequent features:
Attributes become available not only for the current feature but also for previous/subsequent features that have been exposed. As with the current attribute, double-clicking an adjacent attribute adds it to the expression window:
In the above screenshot, the workspace author is finding the time difference between two adjacent records in a business license dataset.
You can see that prior attribute values can be accessed simply by using feature[-x].
Missing Values
The AttributeCreator and AttributeManager also have an option to specify what should happen if the attributes being used in a string are missing:
When the transformer tries to use a value that is missing (or null or empty) this option lets the user choose a replacement value, or to carry out no substitution.
Notice that this setting applies to attributes of the current feature, just as much as attributes of adjacent features.
2) Renaming and Copying Attributes
Renaming and copying attributes are also key attribute functions within FME. When an attribute is renamed it ceases to exist under its prior name; when it is copied it exists both in its new and old names.
The transformers capable of renaming an attribute are:
Task | Transformers |
AttributeCopier | Copy |
AttributeCreator | Copy |
AttributeManager | Copy and Rename |
AttributeRenamer | Rename |
Renaming
The fundamental purpose of renaming is to enter a new name for a selected attribute manually. The old attribute is removed and replaced with the newly named one:
Here an AttributeManager is used to rename a number of fields by entering a different name for the Output Attribute. The Action is automatically set to Rename. Notice that the user is also entering a new constant value for the PSTLCITY/PostalCity attribute.
This type of behavior is obviously of use when the reader schema ('what we have') needs to be renamed to match the writer schema ('what we want').
Tip:
Although you can manually type a new attribute name into the Output Attribute field, if the transformer is connected to a writer feature type with the correct attributes, its attribute names will be automatically available to select from:
Copying
Depending on the transformer, copying an attribute can be one of two styles.
Here the AttributeCopier consists of selecting the existing attribute and entering a new name for it. Again, when connected to a writer feature type, its schema is available to use.
Note how both PSTLCITY and PostalCity exist on the output of the transformer, proving that it is copying the attribute rather than renaming it.
For other transformers, the setup style is reversed: a new attribute is created and given the value of an existing attribute:
In this AttributeManager transformer, the user creates a new attribute (PostalCity) and assigns it the value from another (PSTLCITY). In effect, they have made a copy of the original attribute.
3) Bulk Attribute Renaming
Usual attribute renaming involves selecting individual attributes to modify. However, in some cases, it's important to be able to carry out the same renaming operation on a large number of attributes.
This scenario is catered for by the BulkAttributeRenamer transformer.
BulkAttributeRenamer
The BulkAttributeRenamer carries out the core function of renaming attributes. But instead of manually specifying each attribute, this transformer lets the user select multiple attributes - or all of them:
When multiple attributes are selected, the action must - of course - carry out the same renaming action on them all. These actions are:
- Add String Prefix
- Add String Suffix
- Remove Prefix String
- Remove Suffix String
- Regular Expression Replace
- String Replace
- Change Case
The power of the transformer is also in its ability to manipulate multiple attributes at once, without having to select them all individually. Here, for example, the incoming attributes are all being renamed to lower case names to match a writer schema that does not support upper case:
Multiple transformers can be used to create a cumulative effect. Here, for example, the user has converted to lower case and then used a second transformer to add a prefix:
4) Removing Attributes
Removing attributes is perhaps seen as a less important task in FME. That's because - for a manual attribute schema - only attributes defined in the writer are written to the output; extra attributes that are not required are just ignored.
However, removing attributes does carry useful benefits:
- Removing attributes that aren’t required tidies up a workspace and makes it easier to understand
- A workspace is a complex network of objects and schemas. Removing attributes simplifies this network and makes the Workbench interface more responsive
- All data processing incurs costs of time and memory. Removing attributes means less data is being processed and so the FME engine performs faster
Transformers that can remove attributes are:
- AttributeKeeper
- AttributeManager
- AttributeRemover
- BulkAttributeRemover
The AttributeManager and AttributeRemover have the same technique; select an attribute to be removed:
Attributes can be removed in the AttributeManager by selecting it and clicking the - button. Alternatively, you can change the action field from Do Nothing to Remove.
Notice in the above that three attributes have been removed. The output attribute (when selected) shows the name struck out to signify that it is no longer present.
5) Keeping Attributes
The AttributeKeeper transformer carries out the same function but approaches it from the opposite direction. It lets the user specify which attributes are not to be removed; in other words, this transformer lets the user specify which ones to keep.
So, the AttributeManager should be used where one or two attributes are to be removed, but the majority of them kept. The AttributeKeeper should be used when the majority of attributes are to be removed, and only one or two of them retained.
Bulk Attribute Removal
The BulkAttributeRemover - like the BulkAttributeRenamer - lets the user carry out a process on multiple attributes. In this case, instead of being able to select all attributes, the user enters a string-matching expression to define which attributes to remove:
Here the user is removing all attribute whose name ends in the word "Count."
Transformer List for Category: Attributes
No comments:
Post a Comment